Generalized Linear Model

Overview

A generalized linear model (GLM) extends ordinary linear regression so that the outcome can be continuous, binary, count, or another exponential-family outcome.

Core idea

A GLM models the expected outcome through a link function applied to a linear predictor:

g (E [Y_{i}]) = η_{i} = α + β_{1} X_{i 1} + β_{2} X_{i 2} + \dots

where $g (\cdot)$ is the link function and $η_{i}$ is the linear predictor.

Three Components

Random component: the outcome distribution, e.g. Normal, Bernoulli, Poisson.
Systematic component: the linear predictor $η = X β$ .
Link function: maps the expected outcome to the linear predictor scale.

In other words, a GLM combines:

a linear predictor: $X β$
a link function / nonlinearity: $g (μ)$
a noise model / outcome distribution: $Y \sim$ chosen family

Common GLM Examples

Model	Outcome type	Distribution	Link function	Typical question
Linear regression (Regression#Linear Regression)	Continuous	Normal	Identity	How does $Y$ change with $X$ ?
Logistic regression (Regression#Logistic Regression)	Binary	Bernoulli	Logit	How does $X$ affect the probability of an event?
Poisson regression	Count	Poisson	Log	How does $X$ affect an event rate or count?

Linear Regression as a GLM

Y_{i} \sim Normal (μ_{i}, σ)

μ_{i} = α + β_{X} X_{i} + β_{Z} Z_{i}

Link: identity
Interpretation: coefficients are additive changes in the expected value of $Y$ .

Logistic Regression as a GLM

Y_{i} \sim Bernoulli (p_{i})

logit (p_{i}) = \log (\frac{p_{i}}{1 - p_{i}}) = α + β_{X} X_{i} + β_{Z} Z_{i}

Link: logit
Interpretation: coefficients are changes in log-odds; exponentiated coefficients are odds ratio.
- The odds ratio is the ratio of two groups' odds of some outcome:

o d d s r a t i o = \frac{p_{a}}{1 - p_{a}} / \frac{p_{b}}{1 - p_{b}}

- Odds ratios can be derived from logistic coefficients:

β_{x} = l o g i t (p_{a}) - l o g i t (p_{b}) = l n (o d d s r a t i o)

Poisson Regression as a GLM

Y_{i} \sim Poisson (λ_{i})

\log (λ_{i}) = α + β_{X} X_{i} + β_{Z} Z_{i}

Equivalently:

λ_{i} = \exp (α + β_{X} X_{i} + β_{Z} Z_{i})

Link: log
Interpretation: coefficients are changes in the log expected count; exponentiated coefficients are rate ratios.

Relationship to Other Models

Relationship to Mixed Models

see Regression#

Relationship to GEE

Generalized estimating equations (GEE) extend the GLM idea to correlated observations, such as repeated measurements from the same patient in Longitudinal Data Analysis.

Feature	GLM	GEE
Data	Independent observations	Correlated or repeated observations
Target	Covariate–outcome association	Population-average association
Outcomes	Continuous, binary, count, etc.	Same outcome families via link functions
Correlation	Usually assumes independence	Specifies a working within-subject correlation
Interpretation	Model-specific mean association	Marginal / population-average association

GEE is closely related to GLMs, but it is not simply another GLM distribution. It is an estimation framework for correlated data.

Regression — linear and logistic regression foundations
Linear Mixed Model — mixed-effects models for continuous repeated or clustered outcomes
Longitudinal Data Analysis — choosing among LMM, GEE, and other longitudinal methods
ANOVA & Post-hoc Tests — classical linear-model comparisons
Random Variables & Probability distributions — distributions used as GLM outcome families